Autonomous full spectrum biometric monitoring

ABSTRACT

A device may obtain raw heartbeat data associated with a plurality of wavelength channels. The device may generate, based on a feature vector transformation, a plurality of feature vectors, each corresponding to a respective one of the plurality of wavelength channels. The device may identify a set of selected feature vectors, from the plurality of feature vectors, based on a plurality of squares of correlation coefficients, each associated with a respective pair of the plurality of feature vectors. The device may generate, based on a principal component analysis, an average feature vector of the set of selected feature vectors. The device may determine initial heartbeat cycle data based on the average feature vector. The device may correct heartbeat cycle gaps in the initial heartbeat cycle data in order to determine final heartbeat cycle data.

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/210,740, filed Dec. 5, 2018, which is incorporated herein byreference in its entirety.

BACKGROUND

Photoplethysmography (PPG) is an optical technique that can be used todetect volumetric changes in blood in peripheral circulation (as bloodvolume changes due to the pumping action of the heart). PPG is anon-invasive method that makes measurements at the surface of the skin(e.g., at a fingertip, a wrist, an ear lobe, and/or the like). A PPGdevice may take the form of a multispectral sensor device (e.g., abinary multispectral (BMS) sensor device) that provides heartbeattime-series data associated with multiple wavelength channels (e.g., 64wavelength channels). The multispectral sensor device includes multiplesensor elements (e.g., optical sensors, spectral sensors, and/or imagesensors), each to receive one of the multiple wavelength channels (via arespective region of a multispectral filter) in order to capture theheartbeat time-series data.

SUMMARY

According to some implementations, a method may include: obtaining, by adevice, raw heartbeat data associated with a plurality of wavelengthchannels; generating, by the device and based on a feature vectortransformation, a plurality of feature vectors, each corresponding to arespective one of the plurality of wavelength channels; identifying, bythe device, a set of selected feature vectors, from the plurality offeature vectors, based on a plurality of squares of correlationcoefficients, each associated with a respective pair of the plurality offeature vectors; generating, by the device and using a principalcomponent analysis, an average feature vector of the set of selectedfeature vectors; determining, by the device, initial heartbeat cycledata based on the average feature vector; and correcting, by the device,heartbeat cycle gaps in the initial heartbeat cycle data in order todetermine final heartbeat cycle data to permit a biometric monitoringaction to be performed.

According to some implementations, a device may include one or morememories, and one or more processors, communicatively coupled to the oneor more memories, to: obtain raw heartbeat data associated with aplurality of wavelength channels; generate, based on a feature vectortransformation, a plurality of feature vectors, each corresponding to arespective one of the plurality of wavelength channels; identify a setof selected feature vectors, from the plurality of feature vectors,based on a plurality of squares of correlation coefficients, eachassociated with a respective pair of the plurality of feature vectors;generate, based on a principal component analysis, an average featurevector of the set of selected feature vectors; determine initialheartbeat cycle data based on the average feature vector; and correctheartbeat cycle gaps in the initial heartbeat cycle data in order todetermine final heartbeat cycle data to permit a biometric monitoringaction to be performed.

According to some implementations, a non-transitory computer-readablemedium may store instructions, the instructions including one or moreinstructions that, when executed by one or more processors, cause theone or more processors to: obtain raw heartbeat data associated with aplurality of wavelength channels; generate, based on a feature vectortransformation, a plurality of feature vectors, each corresponding to arespective one of the plurality of wavelength channels; identify a setof selected feature vectors, from the plurality of feature vectors,based on a plurality of squares of correlation coefficients, eachassociated with a respective pair of the plurality of feature vectors;generate, based on a principal component analysis, an average featurevector of the set of selected feature vectors; determine initialheartbeat cycle data based on the average feature vector; and correctheartbeat cycle gaps in the initial heartbeat cycle data in order todetermine final heartbeat cycle data to permit a biometric monitoringaction to be performed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are diagrams of an example implementation describedherein.

FIG. 2 is a diagram of an example environment in which systems and/ormethods described herein may be implemented.

FIG. 3 is a diagram of example components of one or more devices of FIG.2.

FIG. 4 is a flow chart of an example process for determining heartbeatcycle data based on raw heartbeat data associated with a plurality ofwavelength channels.

FIGS. 5A-5D are diagrams illustrating examples of feature vectorsgenerated based on raw heartbeat data, as described herein.

FIG. 6 is a diagram illustrating an example of an average feature vectorgenerated based on a principal component analysis of a set of selectedfeature vectors, as described herein.

FIGS. 7A and 7B are diagrams illustrating an example effect of heartbeatdata gap correction in initial heartbeat cycle data, as describedherein.

FIG. 8 is a diagram illustrating an example of instantaneous heart ratesdetermined based on final heartbeat cycle data, as described herein.

DETAILED DESCRIPTION

The following detailed description of example implementations refers tothe accompanying drawings. The same reference numbers in differentdrawings may identify the same or similar elements.

As described above, a multispectral sensor device may be capable ofmeasuring, obtaining, collecting, or otherwise determining heartbeattime-series data associated with multiple (e.g., 16, 32, 64, and/or thelike) wavelength channels. Such data is herein referred to as rawheartbeat data. In practice the raw heartbeat data can be quite noisy,and can include frequent baseline shifts. Due to such noise and/orbaseline shifts, segmenting the raw heartbeat data into systolic phases(e.g., times during which the heart is contracting) and diastolic phases(e.g., times during which the heart is resting) can be difficult orimpossible. Thus, it is often difficult or impossible to use the rawheartbeat data in association with performing a biometric monitoringaction, such as performing vital sign monitoring (e.g., determining aninstantaneous heart rate, determining a blood pressure, and/or the like)since the raw heartbeat data may lead to inaccurate, unreliable results.

Some implementations described herein provide a heartbeat cycle datadevice for determining, based on raw heartbeat data collected by amultispectral sensor device, heartbeat cycle data based on which abiometric monitoring action can be performed. More specifically, someimplementations described provide a heartbeat cycle data device capableof processing the raw heartbeat data in order to determine the heartbeatcycle data (e.g., data identifying start and end times of heartbeatcycles), thereby allowing a biometric monitoring action that uses theheartbeat cycle data to provide a comparatively more accurate and/orcomparatively more reliable result (e.g., as compared to using rawheartbeat data in association with performing the biometric monitoringaction).

FIGS. 1A and 1B are diagrams of an example implementation 100 describedherein.

As shown in FIG. 1A, a multispectral sensor device may be positionedrelative to a skin surface of a subject. For example, as shown in FIG.1A, the multispectral sensor device may be a device worn on the wrist ofthe subject. In some implementations, the multispectral sensor devicemay be positioned relative to the skin surface at another location onthe body, such as on a fingertip, an arm, a leg, an ear lobe, and/or thelike. In some implementations, the multispectral sensor device includesa BMS sensing device that operates in, for example, the visible (VIS)spectrum, the near infrared (NIR) spectrum, and/or the like.

As shown by reference number 105, the multispectral sensor device maydetermine (e.g., measure, gather, collect, and/or the like) rawheartbeat data associated with N (N>1) wavelength channels. The rawheartbeat data includes, for each of the N wavelength channels,photometric response data that indicates a blood volume beneath the skinsurface (at the location of the multispectral sensor device) at a giventime point.

As shown by reference number 110, a heartbeat cycle data device mayobtain the raw heartbeat data from the multispectral sensor device. Theheartbeat cycle data device is a device capable of determining heartbeatcycle data based on the raw heartbeat data associated with multiplewavelength channels, as described herein. In some implementations, theheartbeat cycle data device may be integrated with the multispectralsensor device (e.g., in a same package, a same housing, on a same chip,and/or the like). Alternatively, the heartbeat cycle data device may beseparate (e.g., remotely located) from the multispectral sensor device.

In some implementations, the heartbeat cycle data device may obtain theraw heartbeat data in real-time or near real-time (e.g., when themultispectral sensor device is configured to provide the raw heartbeatdata as the multispectral sensor device obtains the raw heartbeat data).Additionally, or alternatively, the heartbeat cycle data device mayobtain the raw heartbeat data based on the multispectral sensor device(e.g., automatically) providing the raw heartbeat data on a periodicbasis (e.g., every one second, every five seconds, and/or the like).Additionally, or alternatively, the heartbeat cycle data device mayobtain the raw heartbeat data from the multispectral sensor device basedon requesting the raw heartbeat data from the multispectral sensordevice.

As shown by reference number 115, the heartbeat cycle data device maygenerate N feature vectors, each corresponding to a respective one ofthe N wavelength channels, based on the raw heartbeat data. For example,the heartbeat cycle data device may perform a feature vectortransformation on items of raw heartbeat data associated with each ofthe N wavelength channels in order to generate N feature vectors.

The systolic and diastolic phases of heartbeat cycles are characterizedby upward sloping and downward sloping portions, respectively, ofsignals representing the raw heartbeat data. Thus, if suchcharacteristics are captured in feature vectors that are generated fromtransforming the raw heartbeat data, signal to noise ratios (SNRs) willbe improved (e.g., as compared with the raw heartbeat data). In someimplementations, moving quarter-period slopes feature vectors may begenerated for this purpose. In some implementations, such a featurevector transformation mitigates noise originated from point-to-pointvariations, while providing sufficient range to capture a number of timesteps before the feature vector changes sign.

In some implementations, a quarter-period moving windows slopes featurevector may be generated using the following equation:

FV=(R _(t) −R ₀)/R ₀

where R_(t) is a photometric response, identified by the raw heartbeatdata, at a current time step and R₀ is a photometric response,identified by the raw heartbeat data, wFV time steps before the currenttime step. Here, wFV is a quarter of an estimated heartbeat cycle periodof the subject (e.g., a typical heartbeat cycle period, an averageheartbeat cycle period, a previously determined heartbeat cycle period,and/or the like, which can be adjustable). Notably, while wFV isdescribed as being a quarter (¼) of the heartbeat cycle period of thesubject, wFV can be different than ¼ of the heartbeat cycle period(e.g., ⅙ of the heartbeat cycle period, ⅕ of the heartbeat cycle period,⅓ of the heartbeat cycle period, ½ of the heartbeat cycle period, and/orthe like), in some implementations. Examples illustrating feature vectorgeneration are described below with regard to FIGS. 5A-5D.

In practice, not all wavelength channels will have an acceptable SNRduring operation of the multispectral sensor device and, as a result,the heartbeat cycle data device may need to filter feature vectorscorresponding to these noisy wavelength channels. In someimplementations, the heartbeat cycle data device may perform suchfiltering based on squares of correlation coefficients of the N featurevectors. A lower square of a correlation coefficient (e.g., closer to 0)indicates that two variables are less correlated, while a higher squareof a correlation coefficient indicates that two variables arecomparatively more highly correlated. Thus, as shown by reference number120 in FIG. 1A, the heartbeat cycle data device may determine squares ofcorrelation coefficients for each pair of the N feature vectors.

As shown by reference number 125, the heartbeat cycle data device mayidentify M (M≤N) selected feature vectors based on the squares of thecorrelation coefficients. In some implementations, each of the selectedfeature vectors may correspond to a respective clean wavelength channel.As an example, the heartbeat cycle data device may determine N×(N−1)/2squares of the correlation coefficients, each associated with one pairof the N feature vectors. Next, the heartbeat cycle data device mayidentify a set of M×(M−1)/2 squares of the correlation coefficients, ofthe N×(N−1)/2 determined squares of the correlation coefficients, thatsatisfy a threshold (e.g., a minimum allowable value, such as 0.80,0.90, 0.95, and/or the like). Here, the heartbeat cycle data device mayidentify a set of M selected feature vectors that correspond to the setof M×(M−1)/2 squares of the correlation coefficients that satisfy thethreshold. In this way, the heartbeat cycle data device may filterwavelength channels with an unacceptable amount of noise.

As shown in FIG. 1B, and by reference number 130, after identifying theM selected feature vectors, the heartbeat cycle data device maygenerate, using a principal component analysis (PCA), an average featurevector of the M selected feature vectors. For example, since datasegmentation may be designed to be performed on univariate time-seriesdata, the multivariate time-series data, corresponding to the M selectedfeature vectors, may need to be compressed into univariate time-seriesdata. This compression can be viewed as an averaging of the M selectedfeature vectors. In some implementations, the average feature vector isgenerated based on a first principal component (PC1) determined as aresult of the PCA. In some implementations, generating the averagefeature vector using a PCA allows systolic and diastolic phases ofheartbeats to be more readily identified (e.g., since the averagefeature vector will have a higher SNR than an individual one of the setof selected feature vectors or a mean of the set of selected featurevectors, as described below). An example illustrating such an averagefeature vector is described below with regard to FIG. 6.

As shown by reference number 135, the heartbeat cycle data device maydetermine initial heartbeat cycle data based on the average featurevector. The initial heartbeat cycle data may include information thatidentifies a start time and an end time for an initial set ofheartbeats. In some implementations, the heartbeat cycle data device maydetermine the initial heartbeat cycle data based on the average featurevector. For example, the heartbeat cycle data device may identifypositive phases of the average feature vector (e.g., windows of timeduring which the average feature vector has a substantially positiveslope and crosses a threshold value, such as zero) and negative phasesof the average feature vector (e.g., windows of time during which theaverage feature vector has a substantially negative slope and crossesthe threshold value). In this example, a positive phase (indicating thesystolic phase of a heartbeat) being adjacent to a negative phase(indicating the diastolic phase of the heartbeat) defines one heartbeatcycle period, and start and end times of the heartbeat cycle can bedetermined accordingly. In some implementations, the heartbeat cycledata device may determine initial heartbeat cycle data that identifiesstart and end times for multiple heartbeats.

As further shown in FIG. 1B, and by reference number 140, the heartbeatcycle data device may correct heartbeat cycle gaps in the initialheartbeat cycle data in order to determine final (e.g., gap-free)heartbeat cycle data.

A heartbeat cycle gap may be an error, inaccuracy, inconsistency, and/orthe like, in the initial heartbeat cycle data that results in a givenheartbeat cycle period being identified as significantly shorter orlonger than is reasonably possible. For example, in practice, theaverage feature vector may include spikes and/or noise that canover-divide a heartbeat cycle into several fragments of positive ornegative phases. When this occurs, a heartbeat cycle period identifiedin the initial heartbeat cycle data will be significantly lower than anactual heartbeat cycle period. As a result, a biometric monitoringaction that is performed based on such data may be inaccurate and/orunreliable. For example, an instantaneous heartbeat rate will show asignificant positive spike in the case of an over-divided heartbeat(e.g., such that the determined instantaneous heart rate is much higherthan the actual heart rate).

As another example, a sloping baseline in the average feature vector(e.g., caused by sloping baselines in the raw heartbeat data) can causenegative or positive phases to be undetected. For example, a positivelysloped baseline can cause negative phases to go undetected, and anegatively sloped baseline can cause positive phases to go undetected.When this occurs, a heartbeat cycle period identified in the initialheartbeat cycle data will be significantly higher than an actualheartbeat cycle period. As a result, a biometric monitoring action thatis performed based on such data may be inaccurate and/or unreliable. Forexample, an instantaneous heartbeat rate will have a negative spike inthe case of such undetected phases (e.g., such that the determinedinstantaneous heart rate is much lower than the actual heart rate). Insome implementations, correcting gaps in the initial heartbeat cycledata may remove the effects of over-divided heartbeat cycles and slopingbaselines.

In some implementations, as a first step in correcting heartbeat cyclegaps in the initial heartbeat cycle data, the heartbeat cycle datadevice may identify one or more such heartbeat cycle gaps. For example,the heartbeat cycle data device may determine that a heartbeat cycle,identified by the initial heartbeat cycle data, has a period thatsatisfies a threshold. For example, the heartbeat cycle data device maydetermine that a given heartbeat cycle has a period that is less than aminimum heartbeat cycle period. In such a case, the heartbeat cycle datadevice may identify the presence of a gap caused by an over-dividedheartbeat cycle. As another example, the heartbeat cycle data device maydetermine that a given heartbeat cycle has a period that is greater thana maximum heartbeat cycle period. In such a case, the heartbeat cycledata device may identify the presence of a gap caused by a slopingbaseline.

In some implementations, the heartbeat cycle data device may beconfigured with information that identifies the threshold, and thethreshold may be adjustable (e.g., by the heartbeat cycle data device,by the subject, and/or the like). For example, the heartbeat cycle datadevice may store or have access to information that identifies anestimated heartbeat cycle period associated with the subject (e.g., atypical heartbeat cycle period, an average heartbeat cycle period, apreviously determined heartbeat cycle period, and/or the like), andmaximum allowable positive and negative differences (e.g., an amount oftime, a percentage, and/or the like) from the estimated heartbeat cycleperiod.

In some implementations, after identifying a heartbeat cycle gap, theheartbeat cycle data device may modify the initial heartbeat cycle datain order to correct the identified heartbeat cycle gap. For example, inthe case of an over-divided heartbeat cycle, the heartbeat cycle datadevice may combine adjacent heartbeat cycles until an adjusted heartbeatcycle period is greater than or equal to the minimum heartbeat cycleperiod. As another example, in the case of an undetected phase, theheartbeat cycle data device may identify local troughs or peaks in theaverage feature vector during the heartbeat cycle period, and may dividethe heartbeat cycle into two or more heartbeat cycles (e.g., such thatadjusted heartbeat cycle periods of the two or more heartbeat cycles areless than or equal to the maximum heartbeat cycle period). In someimplementations, the heartbeat cycle data device may correct multipleheartbeat cycle gaps in the initial heartbeat cycle data. In someimplementations, data resulting from the correction of one or moreheartbeat cycle gaps can be referred to as final (e.g., gap-free)heartbeat cycle data. Examples illustrating an effect of heartbeat cyclegap correction are described below with regard to FIGS. 7A and 7B.

In some implementations, the final heartbeat cycle data may permit abiometric monitoring action to be performed (e.g., by the heartbeatcycle data device, by the multispectral sensor device, or by anotherdevice). The biometric monitoring action may include, for example, vitalsign monitoring (e.g., instantaneous heart rate determination, bloodpressure determination, and/or the like), or another type of biometricdetermination and/or monitoring (e.g., blood oxygenation determination,augmentation index determination, hydration determination, and/or thelike).

In some implementations, as shown by reference number 145, the heartbeatcycle data device may provide the final heartbeat cycle data and/orinformation associated with the final heartbeat cycle data. For example,in some implementations, the heartbeat cycle data device may provide thefinal heartbeat cycle data to a device configured to perform vital signmonitoring (e.g., instantaneous heart rate determination, blood pressuredetermination, and/or the like). As another example, in someimplementations, the heartbeat cycle data device may provide the finalheartbeat cycle data to a device configured to perform another type ofbiometric monitoring (e.g., blood oxygen saturation determination,hydration, and/or the like). In some implementations, the finalheartbeat cycle data can be provided for use in a BMS PPG feature matrixbased on which biometric monitoring can be performed.

In some implementations, the heartbeat cycle data device may determinean instantaneous heart rate based on the final heartbeat cycle data, andmay provide (e.g., for display via a display screen of the multispectralsensor device and/or the heartbeat cycle data device) information thatidentifies the instantaneous heart rate. An example associated withdetermination of an instantaneous heart rate is described below withregard to FIG. 8.

In this way, a heartbeat cycle data device can determine heartbeat cycledata based on raw heartbeat data, collected by a multispectral sensordevice, in order to permit a biometric monitoring action to be performedwith increased accuracy and/or increased reliability (e.g., as comparedto performing the biometric monitoring action based on the raw heartbeatdata).

As indicated above, FIGS. 1A and 1B are provided merely as examples.Other examples may differ from what is described with regard to FIGS. 1Aand 1B.

FIG. 2 is a diagram of an example environment 200 in which systemsand/or methods described herein may be implemented. As shown in FIG. 2,environment 200 may include a multispectral sensor device 205, aheartbeat cycle data device 210, and a network 215. Devices ofenvironment 200 may interconnect via wired connections, wirelessconnections, or a combination of wired and wireless connections.

Multispectral sensor device 205 includes a device capable of measuring,gathering, collecting, or otherwise determining raw heartbeat dataassociated with a plurality of wavelength channels, as described herein.For example, multispectral sensor device 205 may include a multispectralsensing device capable of determining raw heartbeat data (in the form ofmultivariate time-series data) on each of 64 wavelength channels. Insome implementations, multispectral sensor device 205 may operate in thevisible spectrum, the near infrared spectrum, the infrared spectrum,and/or the like. In some implementations, multispectral sensor device205 may be a wearable device (e.g., a device worn that can be worn on awrist, a finger, an arm, a leg, a head, an ear, and/or the like). Insome implementations, multispectral sensor device 205 may be integratedwith heartbeat cycle data device 210 (e.g., such that multispectralsensor device 205 and heartbeat cycle data device 210 are on the samechip, in the same package, in the same housing, and/or the like).Alternatively, in some implementations, multispectral sensor device 205may be separate from heartbeat cycle data device 210. In someimplementations, multispectral sensor device 205 may receive informationfrom and/or transmit information to another device in environment 200,such as heartbeat cycle data device 210.

Heartbeat cycle data device 210 includes a device capable of determiningheartbeat cycle data based on raw heartbeat data associated with aplurality of wavelength channels, as described herein. For example,heartbeat cycle data device 210 may include an application specificintegrated circuit (ASIC), an integrated circuit, a server, a group ofservers, and/or the like, and/or another type of communication and/orcomputing device. In some implementations, heartbeat cycle data device210 may be integrated with multispectral sensor device 205 (e.g., suchthat multispectral sensor device 205 and heartbeat cycle data device 210are on the same chip, in the same package, in the same housing, and/orthe like). Alternatively, in some implementations, heartbeat cycle datadevice 210 may be separate from multispectral sensor device 205. In someimplementations, heartbeat cycle data device 210 may receive informationfrom and/or transmit information to another device in environment 200,such as multispectral sensor device 205.

Network 215 includes one or more wired and/or wireless networks. Forexample, network 215 may include a wired network (e.g., whenmultispectral sensor device 205 and heartbeat cycle data device 210 areincluded in same package and/or a same chip). As another example,network 215 may include a cellular network (e.g., a long-term evolution(LTE) network, a code division multiple access (CDMA) network, a 3Gnetwork, a 4G network, a 5G network, another type of next generationnetwork, etc.), a public land mobile network (PLMN), a local areanetwork (LAN), a wide area network (WAN), a metropolitan area network(MAN), a telephone network (e.g., the Public Switched Telephone Network(PSTN)), a private network, an ad hoc network, an intranet, theInternet, a fiber optic-based network, a cloud computing network, or thelike, and/or a combination of these or other types of networks.

The number and arrangement of devices and networks shown in FIG. 2 areprovided as an example. In practice, there may be additional devicesand/or networks, fewer devices and/or networks, different devices and/ornetworks, or differently arranged devices and/or networks than thoseshown in FIG. 2. Furthermore, two or more devices shown in FIG. 2 may beimplemented within a single device, or a single device shown in FIG. 2may be implemented as multiple, distributed devices. Additionally, oralternatively, a set of devices (e.g., one or more devices) ofenvironment 200 may perform one or more functions described as beingperformed by another set of devices of environment 200.

FIG. 3 is a diagram of example components of a device 300. Device 300may correspond to multispectral sensor device 205 and/or heartbeat cycledata device 210. In some implementations, multispectral sensor device205 and/or heartbeat cycle data device 210 may include one or moredevices 300 and/or one or more components of device 300. As shown inFIG. 3, device 300 may include a bus 310, a processor 320, a memory 330,a storage component 340, an input component 350, an output component360, and a communication interface 370.

Bus 310 includes a component that permits communication among thecomponents of device 300. Processor 320 is implemented in hardware,firmware, or a combination of hardware and software. Processor 320 is acentral processing unit (CPU), a graphics processing unit (GPU), anaccelerated processing unit (APU), a microprocessor, a microcontroller,a digital signal processor (DSP), a field-programmable gate array(FPGA), an application-specific integrated circuit (ASIC), or anothertype of processing component. In some implementations, processor 320includes one or more processors capable of being programmed to perform afunction. Memory 330 includes a random access memory (RAM), a read onlymemory (ROM), and/or another type of dynamic or static storage device(e.g., a flash memory, a magnetic memory, and/or an optical memory) thatstores information and/or instructions for use by processor 320.

Storage component 340 stores information and/or software related to theoperation and use of device 300. For example, storage component 340 mayinclude a hard disk (e.g., a magnetic disk, an optical disk, amagneto-optic disk, and/or a solid state disk), a compact disc (CD), adigital versatile disc (DVD), a floppy disk, a cartridge, a magnetictape, and/or another type of non-transitory computer-readable medium,along with a corresponding drive.

Input component 350 includes a component that permits device 300 toreceive information, such as via user input (e.g., a touch screendisplay, a keyboard, a keypad, a mouse, a button, a switch, and/or amicrophone). Additionally, or alternatively, input component 350 mayinclude a sensor for sensing information (e.g., a global positioningsystem (GPS) component, an accelerometer, a gyroscope, and/or anactuator). Output component 360 includes a component that providesoutput information from device 300 (e.g., a display, a speaker, and/orone or more light-emitting diodes (LEDs)).

Communication interface 370 includes a transceiver-like component (e.g.,a transceiver and/or a separate receiver and transmitter) that enablesdevice 300 to communicate with other devices, such as via a wiredconnection, a wireless connection, or a combination of wired andwireless connections. Communication interface 370 may permit device 300to receive information from another device and/or provide information toanother device. For example, communication interface 370 may include anEthernet interface, an optical interface, a coaxial interface, aninfrared interface, a radio frequency (RF) interface, a universal serialbus (USB) interface, a Wi-Fi interface, a cellular network interface, orthe like.

Device 300 may perform one or more processes described herein. Device300 may perform these processes based on processor 320 executingsoftware instructions stored by a non-transitory computer-readablemedium, such as memory 330 and/or storage component 340. Acomputer-readable medium is defined herein as a non-transitory memorydevice. A memory device includes memory space within a single physicalstorage device or memory space spread across multiple physical storagedevices.

Software instructions may be read into memory 330 and/or storagecomponent 340 from another computer-readable medium or from anotherdevice via communication interface 370. When executed, softwareinstructions stored in memory 330 and/or storage component 340 may causeprocessor 320 to perform one or more processes described herein.Additionally, or alternatively, hardwired circuitry may be used in placeof or in combination with software instructions to perform one or moreprocesses described herein. Thus, implementations described herein arenot limited to any specific combination of hardware circuitry and/orsoftware.

The number and arrangement of components shown in FIG. 3 are provided asan example. In practice, device 300 may include additional components,fewer components, different components, or differently arrangedcomponents than those shown in FIG. 3. Additionally, or alternatively, aset of components (e.g., one or more components) of device 300 mayperform one or more functions described as being performed by anotherset of components of device 300.

FIG. 4 is a flow chart of an example process 400 for determiningheartbeat cycle data based on raw heartbeat data associated with aplurality of wavelength channels. In some implementations, one or moreprocess blocks of FIG. 4 may be performed by a heartbeat cycle datadevice (e.g., heartbeat cycle data device 210). In some implementations,one or more process blocks of FIG. 4 may be performed by another deviceor a group of devices separate from or including the heartbeat cycledata device, such as a multispectral sensor device (e.g., multispectralsensor device 205), and/or the like.

As shown in FIG. 4, process 400 may include obtaining raw heartbeat dataassociated with a plurality of wavelength channels (block 410). Forexample, the heartbeat cycle data device (e.g., using processor 320,memory 330, storage component 340, input component 350, communicationinterface 370, and/or the like) may obtain raw heartbeat data associatedwith a plurality of wavelength channels, as described above.

As shown in FIG. 4, process 400 may include generating, based on afeature vector transformation, a plurality of feature vectors, eachcorresponding to a respective one of the plurality of wavelengthchannels (block 420). For example, the heartbeat cycle data device(e.g., using processor 320, memory 330, and/or the like) may generate,based on a feature vector transformation, a plurality of featurevectors, each corresponding to a respective one of the plurality ofwavelength channels, as described above.

As shown in FIG. 4, process 400 may include identifying a set ofselected feature vectors, from the plurality of feature vectors, basedon a plurality of squares of correlation coefficients, each associatedwith a respective pair of the plurality of feature vectors (block 430).For example, the heartbeat cycle data device (e.g., using processor 320,memory 330, and/or the like) may identify a set of selected featurevectors, from the plurality of feature vectors, based on a plurality ofsquares of correlation coefficients, each associated with a respectivepair of the plurality of feature vectors, as described above.

As shown in FIG. 4, process 400 may include generating, using aprincipal component analysis, an average feature vector of the set ofselected feature vectors (block 440). For example, the heartbeat cycledata device (e.g., using processor 320, memory 330, and/or the like) maygenerate, using a principal component analysis, an average featurevector of the set of selected feature vectors, as described above.

As shown in FIG. 4, process 400 may include determining initialheartbeat cycle data based on the average feature vector (block 450).For example, the heartbeat cycle data device (e.g., using processor 320,memory 330, and/or the like) may determine initial heartbeat cycle databased on the average feature vector, as described above.

As shown in FIG. 4, process 400 may include correcting heartbeat cyclegaps in the initial heartbeat cycle data in order to determine finalheartbeat cycle data to permit a biometric monitoring action to beperformed (block 460). For example, the heartbeat cycle data device(e.g., using processor 320, memory 330, and/or the like) may correctheartbeat cycle gaps in the initial heartbeat cycle data in order todetermine final heartbeat cycle data, to permit a biometric monitoringaction to be performed, as described above.

Process 400 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the biometric monitoring action isinstantaneous heart rate determination. Here, the heartbeat cycle datadevice may determine an instantaneous heart rate based on the finalheartbeat cycle data, and provide information that identifies theinstantaneous heart rate, in some implementations.

In some implementations, the biometric monitoring action is vital signmonitoring. Here, the heartbeat cycle data device may provide the finalheartbeat cycle data in association with performing the vital signmonitoring.

In some implementations, the plurality of feature vectors is a pluralityof moving quarter-period slopes feature vectors.

In some implementations, the heartbeat cycle data device may determinethe plurality of squares of the correlation coefficients for theplurality of feature vectors, and identify a set of squares of thecorrelation coefficients, of the plurality of squares of the correlationcoefficients, that satisfy a threshold. Here, when identifying the setof selected feature vectors, the heartbeat cycle data device mayidentify the set of selected feature vectors based on the set of squaresof the correlation coefficients, where each of the set of squares of thecorrelation coefficients corresponds to a respective pair of the set ofselected feature vectors.

In some implementations, the average feature vector is generated basedon a first principal component associated with the principal componentanalysis.

In some implementations, when correcting the heartbeat cycle gaps inorder to determine the final heartbeat cycle data, the heartbeat cycledata device may identify a heartbeat cycle gap based on determining thata heartbeat cycle, identified by the initial heartbeat cycle data, has aperiod that satisfies a threshold; and modify, based on an estimatedheartbeat cycle period, the initial heartbeat cycle data in order tocorrect the identified heartbeat cycle gap, where a result of modifyingthe initial heartbeat cycle data to correct the identified heartbeatcycle gap is the final heartbeat cycle data.

Although FIG. 4 shows example blocks of process 400, in someimplementations, process 400 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 4. Additionally, or alternatively, two or more of theblocks of process 400 may be performed in parallel.

FIGS. 5A-5D are diagrams illustrating examples of feature vectorsgenerated based on raw heartbeat data, as described herein. Notably,identification of individual signals is not necessary for understandingthe implementation illustrated by FIGS. 5A-5D; consequently, cleardelineation of each signal is not shown.

FIG. 5A is an example illustrating raw heartbeat data (in the form ofrelative count change of average counts) for 14 wavelength channelsduring a time window from time step 0 to approximately time step 180.Notably, the raw heartbeat data illustrated in FIG. 5A has a relativelyflat baseline. In some implementations, as described above, featurevectors can be generated for each of the wavelength channels. FIG. 5B isan example illustrating a result of transforming the raw heartbeat data,associated with each of the 14 wavelength channels, to a respectivemoving quarter-period slopes feature vector. As shown in FIG. 5B,positive and negative phases of the feature vectors are readilyidentifiable (e.g., as compared to the raw heartbeat data). Thus, asdescribed above, feature vector transformation may allow heartbeat cycledata to be determined by heartbeat cycle data device 210.

FIG. 5C is an example illustrating raw heartbeat data (in the form ofrelative count change of average counts) for the 14 wavelength channelsduring a time window from approximately time step 180 to approximatelytime step 320. Notably, the raw heartbeat data illustrated in FIG. 5Chas a baseline with a significantly positive slope. In someimplementations, as described above, feature vectors can be generatedfor each of the wavelength channels. FIG. 5D is an example illustratinga result of transforming the raw heartbeat data, associated with each ofthe 14 wavelength channels, to a respective moving quarter-period slopesfeature vector. As shown in FIG. 5D, positive and negative phases of thefeature vectors are relatively identifiable (e.g., as compared to theraw heartbeat data). Thus, as described above, feature vectortransformation may allow heartbeat cycle data to be determined byheartbeat cycle data device 210. In some implementations, as describedabove, heartbeat cycle data device 210 may correct gaps resulting fromthe positively sloped baseline in the manner described above (e.g.,after generating an average feature vector).

As indicated above, FIGS. 5A-5D are provided merely as examples. Otherexamples may differ from what is described with regard to FIGS. 5A-5D.

FIG. 6 is a diagram illustrating an example of an average feature vectorgenerated based on a principal component analysis of a set of selectedfeature vectors, as described herein. As shown in FIG. 6, in thisexample, a feature vector corresponding to a 506 nanometer (nm)wavelength channel, a feature vector corresponding to a 520 nmwavelength channel, and a feature vector corresponding to a mean of theset of selected feature vectors (e.g., a set of selected feature vectorsthat includes the feature vectors corresponding to the 506 nm and 520 nmchannels) have values that range between approximately −0.02 andapproximately 0.02 in a given time window. As further shown, an averagefeature vector (PC1) generated using a PCA of the set of selectedfeature vectors has a value that ranges between approximately −0.06 and0.08 in the given time window.

As illustrated by FIG. 6, generating the average feature vector (e.g.,PC1) using a PCA provides a signal with a significantly higher SNR(e.g., as compared to an individual one of the set of selected featurevectors or a mean of the set of selected feature vectors) and,therefore, allows positive and negative phases to be more readilyidentified in association with determining heartbeat cycle data in themanner described above.

As indicated above, FIG. 6 is provided merely as an example. Otherexamples may differ from what is described with regard to FIG. 6.

FIGS. 7A and 7B are diagrams illustrating an example effect of heartbeatdata gap correction in initial heartbeat cycle data, as describedherein. FIG. 7A illustrates an example of numbers of occurrences ofdifferent heartbeat periods before and after gap correction during agiven time window, and FIG. 7B illustrates an example of determinedinstantaneous heart rates before and after gap correction during thegiven time window.

As shown in FIG. 7A, before gap correction, a number of over-dividedheartbeat cycle periods (e.g., heartbeat cycle periods with periods ofless than approximately 20 time steps) and undetected heartbeat cycleperiods are present. As further shown, after gap correction, these gapsare removed (e.g., such that all heartbeat cycles have a period that isbetween approximately 20 and approximately 32 time steps).

Similarly, as shown in FIG. 7B, before gap correction, the instantaneousheart rate includes numerous positive spikes (e.g., above approximately150 beats per minute) resulting from the over-divided heartbeat cycleperiods, and negative spikes resulting from the undetected heartbeatcycle periods. As further shown, after gap correction, these spikes areremoved (e.g., such that the instantaneous heart rate does not includesudden significant increases or decreases).

As indicated above, FIGS. 7A and 7B are provided merely as examples.Other examples may differ from what is described with regard to FIGS. 7Aand 7B.

FIG. 8 is a diagram illustrating an example of instantaneous heart ratesdetermined based on final heartbeat cycle data, a described herein. Asshown in FIG. 8, an instantaneous heart rate determined based on finalheartbeat cycle data, as described herein, may approximately match(e.g., within a few beats per minute) an instantaneous heart ratedetermined in a conventional chest strap heart rate monitor (which isknown to be relatively accurate). Notably, further improvement mayresult from smoothing of the determined instantaneous heart rate (e.g.,using a moving window average) and, therefore, a closer match than thatillustrated in FIG. 8 can be achieved. In either case, as illustrated inFIG. 8, the final heartbeat cycle data, determined in the mannerdescribed herein, may allow for a biometric monitoring action to beperformed with an acceptable accuracy and/or reliability.

As indicated above, FIG. 8 is provided merely as an example. Otherexamples may differ from what is described with regard to FIG. 8.

Some implementations described herein allow heartbeat cycle data device210 to determine, based on raw heartbeat data collected by multispectralsensor device 205, heartbeat cycle data based on which a biometricmonitoring action can be performed. More specifically, someimplementations described herein allow heartbeat cycle data device 210to process the raw heartbeat data in order to determine the heartbeatcycle data, thereby allowing a biometric monitoring action that uses theheartbeat cycle data to provide a comparatively more accurate and/orcomparatively more reliable result (e.g., as compared to using rawheartbeat data in association with performing the biometric monitoringaction).

The foregoing disclosure provides illustration and description, but isnot intended to be exhaustive or to limit the implementations to theprecise form disclosed. Modifications and variations may be made inlight of the above disclosure or may be acquired from practice of theimplementations.

As used herein, the term “component” is intended to be broadly construedas hardware, firmware, or a combination of hardware and software.

Some implementations are described herein in connection with thresholds.As used herein, satisfying a threshold may refer to a value beinggreater than the threshold, more than the threshold, higher than thethreshold, greater than or equal to the threshold, less than thethreshold, fewer than the threshold, lower than the threshold, less thanor equal to the threshold, equal to the threshold, or the like,depending on the context.

It will be apparent that systems and/or methods, described herein, maybe implemented in different forms of hardware, firmware, or acombination of hardware and software. The actual specialized controlhardware or software code used to implement these systems and/or methodsis not limiting of the implementations. Thus, the operation and behaviorof the systems and/or methods are described herein without reference tospecific software code—it being understood that software and hardwarecan be designed to implement the systems and/or methods based on thedescription herein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of various implementations. In fact,many of these features may be combined in ways not specifically recitedin the claims and/or disclosed in the specification. Although eachdependent claim listed below may directly depend on only one claim, thedisclosure of various implementations includes each dependent claim incombination with every other claim in the claim set.

No element, act, or instruction used herein should be construed ascritical or essential unless explicitly described as such. Also, as usedherein, the articles “a” and “an” are intended to include one or moreitems, and may be used interchangeably with “one or more.” Furthermore,as used herein, the term “set” is intended to include one or more items(e.g., related items, unrelated items, a combination of related andunrelated items, etc.), and may be used interchangeably with “one ormore.” Where only one item is intended, the phrase “only one” or similarlanguage is used. Also, as used herein, the terms “has,” “have,”“having,” or the like are intended to be open-ended terms. Further, thephrase “based on” is intended to mean “based, at least in part, on”unless explicitly stated otherwise.

What is claimed is:
 1. A method, comprising: receiving, by a device, rawheartbeat data from a multispectral sensor device; identifying, by thedevice and based on the raw heartbeat data, a plurality of featurevectors; identifying, by the device, one or more selected featurevectors, from the plurality of feature vectors, based on a plurality ofsquares of correlation coefficients associated with the plurality offeature vectors; determining, by the device, final heartbeat cycle databased on the one or more selected feature vectors; and providing, by thedevice, information associated with the final heartbeat cycle data. 2.The method of claim 1, wherein the multispectral sensor device ispositioned relative to a skin surface of a subject.
 3. The method ofclaim 1, wherein the device is integrated with the multispectral sensordevice in a same housing.
 4. The method of claim 1, wherein the deviceis remotely located from the multispectral sensor device.
 5. The methodof claim 1, wherein a feature vector, of the one or more selectedfeature vectors, corresponds to a wavelength channel.
 6. The method ofclaim 1, wherein a square of correlation coefficients, of the pluralityof squares of correlation coefficients, is associated with a pair of theplurality of feature vectors.
 7. The method of claim 1, whereinidentifying the one or more selected feature vectors comprises:identifying the one or more selected feature vectors, from the pluralityof feature vectors, that correspond to a set of squares, of theplurality of squares of correlation coefficients, that satisfy athreshold.
 8. A device, comprising: one or more memories; and one ormore processors, coupled to the one or more memories, configured to:identify, based on raw heartbeat data, a plurality of feature vectors;identify one or more selected feature vectors, from the plurality offeature vectors, based on a plurality of squares of correlationcoefficients associated with the plurality of feature vectors; determinefinal heartbeat cycle data based on the one or more selected featurevectors; and provide information associated with the final heartbeatcycle data.
 9. The device of claim 8, wherein the one or more processorsare further configured to: receive raw heartbeat data from amultispectral sensor device that is positioned relative to a skinsurface of a subject.
 10. The device of claim 8, wherein the one or moreprocessors are further configured to: receive raw heartbeat data from amultispectral sensor device that is integrated with the device in a samehousing.
 11. The device of claim 8, wherein the one or more processorsare further configured to: receive raw heartbeat data from amultispectral sensor device that is remotely located from the device.12. The device of claim 8, wherein a feature vector, of the one or moreselected feature vectors, corresponds to a wavelength channel.
 13. Thedevice of claim 8, wherein a square of correlation coefficients, of theplurality of squares of correlation coefficients, is associated with apair of the plurality of feature vectors.
 14. The device of claim 8,wherein the one or more processors, to identify the one or more selectedfeature vectors, are configured to: identify the one or more selectedfeature vectors, from the plurality of feature vectors, that correspondto a set of squares, of the plurality of squares of correlationcoefficients, that satisfy a threshold.
 15. A non-transitorycomputer-readable medium storing a set of instructions, the set ofinstructions comprising: one or more instructions that, when executed byone or more processors of a device, cause the device to: receive rawheartbeat data from a multispectral sensor device; identify, based onthe raw heartbeat data, a plurality of feature vectors; identify one ormore selected feature vectors, from the plurality of feature vectors,based on the plurality of feature vectors; determine heartbeat cycledata, based on the one or more selected feature vectors; and provideinformation associated with the final heartbeat cycle data.
 16. Thenon-transitory computer-readable medium of claim 15, wherein themultispectral sensor device is positioned relative to a skin surface ofa subject.
 17. The non-transitory computer-readable medium of claim 15,wherein the device is integrated with the multispectral sensor device ina same housing.
 18. The non-transitory computer-readable medium of claim15, wherein the device is remotely located from the multispectral sensordevice.
 19. The non-transitory computer-readable medium of claim 15,wherein a feature vector, of the one or more selected feature vectors,corresponds to a wavelength channel.
 20. The non-transitorycomputer-readable medium of claim 15, wherein the one or more selectedfeature vectors are identified based on a square of correlationcoefficients associated with a pair of the plurality of feature vectors.